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INTRODUCTION 


This guide is one of the products of a 1979 summer grant from 
the Secretary of State. It represents the first in a series of Social 
Planning and Research Council publications for community service managers. 


It is designed to assist managers in utilizing their client 
record keeping systems es an evaluative resource, It represents one 
effort to bridge the gap between evaluative researchers and community 
service practitioners. 


Despite the significant developments which have occurred in 
evaluation research during the past decade, there is a notable lack of 
trained evaluation researchers to assist administrators in the evaluation 
of their programs. At the same time, the manager is under considerable 
pressure from funders to implement program evaluation procedures. 


This situation presents an obvious problem for community 
service managers insofar as they are being forced to develop evaluative 
systems without access to the required technical resources. As the 
demands for accountability increase, this problem will become more acute. 


This publication will not attempt to turn every manager into 
an evaluation researcher, It will, however, attempt to present the 
manager with a general approach to evaluating their service. Specifically, 
it will demonstrate how client record systems can be used as an evaluative 
tool, 


It is very important that we receive feedback about the adequacy 
of this publication, Given this feedback, we can revise future editions 
of this publication, as well as plan other publications in our community 
service management series. The last page of this guide is a questionnaire 
which we hope you will complete after you have read and, hopefully, used 
the guide, Please complete it and return it to the Research Director, 


Digitized by the Internet Archive 
in 2023 with funding from 
Hamilton Public Library 


https://archive.org/details/developingselfevOOunse 


I. SELF EVALUATION AND ACCOUNTABILITY 


No one working in the community services system needs to be 
told of the growing importance of accountability. Community service 
managers are increasingly being required to justify their services in 


terms which can be understood by funders and the public at large. 


Ary funded service has an obvious financial accountability 
insofar as it must account somehow for the money which it is spending. 
The standard agency budget, in audited form, performs this function in 
the traditional sense. At least it testifies to the fact that agency 
personnel are not absconding with the funds, As community service 
facilities have grown more complex, however, and the number of programs 
have multiplied, managers have been forced to go one level beyond the 
standard agency budget to program budgetting or, as it is sometimes 


called, functional budgetting. 


The other area of accountability is, of course, program 
accountability. That is, given that the money is being spent as planned, 
are the resulting services effectively meeting the needs as planned? It 
is in this area, that of effectiveness, that program evaluation has been 


receiving increased attention. 


When these two areas of accountability - financial and effec- 
tiveness - come together, then the issue is one of efficiency. Are 
these effective services being offered to the largest number of people 


at the lowest possible cost? 


THE EVALUATIVE QUESTION 


The present paper is primarily concerned with the effective- 
ness dimension and, consequently, program evaluation. Evaluative 
research, like all research, is concerned with the generation of 
information to answer certain questions and the typical evaluation 


question, as mentioned above, is - Is this program effective? 
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Experience has shown that, in general, that question is a poor 
one insofar as it begs a yes-or-no answer. Such an answer is rarely 


possible for a number of reasons. 


Firstly, success is a very relative thing. Is a success rate 
of 30% a good or a bad success rate? This judgement depends upon at 
least three factors - what are the success rates of other programs; 
what kind of clients are being served; and what kind of staff resources 


is the agency able to invest to secure that success rate. 


Secondly, any program is likely to be more successful with 
some kind of clients than others. Given a general lack of appropriate 
research, an agency does not even necessarily know what kind of clients 
they are most successful with, Additionally, given the mandate of most 
agencies, they are not necessarily able to restrict their intake to 
those clients having a better prognosis, Some managers, in fact, would 


argue loudly that such a restriction would be improper. 


Following upon this point, Hayes-Roth et al (1972)* concluded 
that the essential question for evaluative research in the field of 
psychotherapy is ~ "What kind of patient problem is helped by what kind 
of treatment, for what kind of patient?", This is the evaluative 
question which will form the basis of this paper. If, at first reading, 
the question sounds somewhat confusing, it will be clarified and 


discussed in more depth later, 


Given a workable question, the next issue is, of course, who 
should answer it? There has been considerable debate about this issue 
in the evaluation literature. Should evaluations be carried out by the 
agency, the funder or a third party? This depends, not surprisingly, 


upon the purpose of the evaluation itself. 


a ee ee 
*Hayes-Roth, F,; Longabaugh, R.; Ryback, S. "The Problem-oriented 


medical record and psychiatry", British Journal of Psychiatry, 1972, 
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In the context of this paper, we are assuming that the agency 


should be evaluating itself. That is why we speak of self-evaluation. 


Funder and third party evaluations are typically one-shot 
efforts désigned, "once and for all" in a safe scientific manner, to 
indicate whether this program is effective or not. As reflected in the 
previous statement, they usually begin with the wrong question. As one- 
shot efforts, they are of limited utility to agency personnel who require 
more of a continuous monitoring type of process if they intend to use the 
results for making administrative decisions. Thirdly, evaluations must, 
to a certain extent, be tailored to the specific agency and, subsequently, 
evolve with the agency. Funder and third party evaluations rarely have 
this quality and, therefore, justify the manager's complaint that the 


results are sometimes irrelevant. 


Our position, therefore, is that, although funder and third 
party evaluations do have a place, highest priority should be attached 
to the development of internal, on-going self-evaluative systems in 
agencies, Certainly, the funder and third party evaluation will dominate 


evaluation research until agencies are capable of self-evaluation. 


Quantitative evaluations require a data-base from which the 
information can be generated. The data base within an agency is, of 
course, the client records. Unfortunately, most agencies do not have 
the kind of client records which are conducive to an adequate self- 


evaluation. 


The purpose of this paper is to assist community service 
managers to develop an adequate information base for agency self- 


evaluation. 
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II, THE INFORMATION SYSTEM 


The term "information systems" brings to mind, in many 
professionals, the prospect of expensive and complicated computer 
systems. This need not be the case. In fact, in many agency applica- 
tions, the use of a computer would clearly be a case of "overkill" - a 


manual system would be much more appropriate, 


All of the concepts discussed in this paper can be utilized 


in either a manual or automated computer system. 


WHAT IS AN INFORMATION SYSTEM? 


An information system is simply a collection of information 
which is continually updated (input) and utilized for some purpose 
(output). The information system which we will be discussing in this 


paper is statistically—based and, thus, we speak of a data base. 


Different information systems differ in terms of their 
complexity and the need for computerization increases with the complex- 
ity. The amount of complexity varies with the amount of information in 


the system and the type of output or reporting which is required of it. 


In developing your own system, keep it simple. If expansion 
is necessary, any information system can be expanded at a future date, 


Start small and start manual. 


WHY HAVE AN INFORMATION SYSTEM? 


This question, although a popular one, is actually redundant. 
Every agency does, in fact, have an information system. It is just a 


question of how that information system is utilized. 


Any agency which keeps client files has an information system. 
These files are probably updated by the individual therapist or worker 
and utilized by them in the course of their day-to-day work. This 
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information may be recorded in a standard fashion with each recorder 
using the same bits or categories of information or it may be done in 


an unstandardized manner (i.e. case notes). 


Such an information system is primarily designed for the use 
of the frontline therapist or worker. It allows them to keep up-to- 
date on their work, refreshes their memory and can provide for the 


"smooth" transfer of a client from one worker to another. 


Its use, however, as an evaluative tool is limited. If one 
is to carry out an evaluation based on this type of system, one is 
almost forced to simply ask each worker's opinion of how successful 


his/her work is. 


To be an effective evaluation tool, an agency information 


system must have the following characteristics, 


1) Standardized input: All of the information entering the 
system must be standardized. That is, 
each therapist or worker must be using 
common terminology to allow for a 
summary of the information. 


2) Continuous input: The information must be constantly 
updated (i.e. new cases added) in order 
to provide relevant summaries. 


3) Continuous output: The summarized information must be 
published on a continual basis (i.e. 
monthly) to allow for an ongoing 
monitoring of agency performance. 


THE COMPONENTS OF AN INFORMATION SYSTEM 


As described above, an information system has three elementa 


components - input, analysis and output. The input is, of course, the 


ry 


information which is put into the system. The most frequent example of 


input is client data. 
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The analysis involves summarizing this information into a 


useful and clearly understood set of documents. 


These documents are the output of the system. Other output 


mediums, particularly when the system is computerized, are available. 


TYPES OF AGENCY INFORMATION 


Most of the information contained in an agency can be 


classified into one of two types - resource data and client data. 


Resource data is the information which describes the resources 
which an agency utilizes to deliver their service. This is expressed 
primarily in terms of finances and staff although, in some larger 
services, equipment and facilities become a major resource. Examples 


of resource data are - number of staff, cost per unit of service, etc. 


Client data is that information which described the recipients 
of the service. This includes demographic information (age, sex, 
marital status, etc.); problem definition data (i.e. psychiatric 
diagnosis) and change data (information relating to the results of the 


service upon the client. 


These two groups of information can be combined at the output 
stage. An example of this type of combination would be the cost/unit 


of service for family counselling clients. 


Most of this guide will be concerned with the utilization of 


client data as opposed to resource data. 


PUTTING IT ALL TOGETHER 


How then does this information relate to the evaluation 
question? For our purpose, we will simplify the evaluation question to 


- "What kind of client is most helped by our service?". 


As presented graphically in Table I each type of agency 


information is relevant to one aspect of the evaluative question. 
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THE 
EVALUATIVE 
QUESTION 


What kind of client 
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TABLE I 


THE 
INFORMATION 
SYSTEM 


Demographics 

Problem definition 
(To describe kinds 
of clients) 


Evaluative Information 


Resource data to 
describe the service 
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III. CLIENT INFORMATION: DEMOGRAPHICS AND PROBLEM DEFINITION 


DEMOGRAPHIC INFORMATION 


Demographic information is the basic situational data which 
almost any agency keeps about their clients - age, sex, marital status, 


employment, educ*tion, etc. 


For ease of analysis in an information system, this data 
should be categorized at the stage of input. Age can be categorized 
into age groups (i.e. 0-5, 16-18, 40-45, etc.) and the precise group- 
ings used will depend upon the nature of the agency. The major 
consideration is to keep the number of categories at a manageable level. 
Thus, an agency which serves only 16 to 21 year olds may still record 
age on a single year basis insofar as they will only end up with six 


categories. 


Sex, marital status and number of dependents or children are 


easily classified. 


A useful classification for employment status is employed; 
unemployed; student; homemaker; inactive. Use the standard definition 
of unemployed - someone who does not have a job but is looking for work. 
as opposed to unemployed persons not looking for work, The latter would 


be classified as inactive. 


Classify the individual's occupation according to the standard 
Statistics Canada occupational listing. This is presented in Appendix 


I of this guide. 


A very useful demographic statistic which is seldom used by 
agencies is "location", Any urban area is divided into census tracts 
by Statistics Canada for use in the National Census. The City of 


Hamilton, for example, is divided into seventy-eight census tracts. 
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A street index is available from Statistics Canada which allows the 


user to very quickly identify the correct census tract for any address. 


Alternatively, large urban areas are often divided into 
planning neighbourhoods or wards, Again, a street index is usually 
available from the area planning department for locating addresses. 

In Hamilton, we are fortunate insofar as the wealth of socio-economic 
data from the National Census has been recoded on the basis of planning 


neighbourhoods. 


Geographical coding allows administrators to monitor where 
demand for their services is coming from. Additionally, by utilizing 
the socio-economic information from the National Census, the service 
planner can identify what kind of areas, in socio-economic terms, have 


a high demand for their particular service. 


To start, however, it would be worth your while to go 
through the appropriate research literature to identify which demo- 
grahic characteristics have been found to be associated with treatment 
success in your service area, To a considerable extent, this can also 
be done intuitively. Make certain that these statistics are included 


in your demographic profile. 


The demographic information kept by seven sample services in 


the local area is presented in Appendix II. 


PROBLEM DEFINITION INFORMATION 


In some standardized manner, the problem of the client has to 
be categorized. There are a number of systems available for doing This. 
and it obviously ‘has to be tailored to the specific needs of the 


agencies. 


Most psychiatric services utilize the standard diagnostic 


categories for mental disorders as developed by the American Psychiatric 
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Association. These are mental retardation, organic brain syndromes, 
non-physical psychoses, neuroses, personality disorders, psycho 


physiologic disorders, etc, and their numerous sub-categories. 


For non-psychiatric social services, there are a number of 
possible systems. Considerable work has been done in this regard in 
the area of "problem-oriented record keeping". One of the more 
complex systems is that of Richard Grant. He devised a "problem coding 
system" which operates at four levels of differentiation and specifies 
over 200 specific problem areas, The Grant system is actually a com- 


bined psychiatric-social service diagnostic scheme. 


The problem definition systems of three local services, by 
way of illustration, are presented in Appendix III of this guide. It 
is obvious when reading these schemas that each reflects the unique 


mandate of each agency. 


Thus, the precise categories have to be tailored to the 
individual agency. Regardless, however, of which categories are used, 
they must be defined in very specific terms to meet the criteria of 


standardization. 


When the problem definitions are used to generate treatment 
goals, then the evaluation process is begun, Such a system is called 


a "goal-oriented records" system. 


CODING THE INFORMATION 


The analysis of this information can be greatly simplified 
if some system of "coding" is a applied. For example, if five age 
categories are used, these can be coded from one through five as in 


Figure I. 


The same coding system can be applied to the various problem- 


definition categories. 
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The importance of this coding system will become more obvious 


in the section dealing with the analysis of the information. 


FIGURE I 
Age Code 
Categories 
16-19 mypt 
90-27, non 
25-29 "3 " 
30-34 al 


35 & over ae 


IV: CLIENT INFORMATION: EVALUATION OF CLIENT CHANGE 


This body of information tends to be the most neglected area 
of agency record keeping because of the general complexity of evaluation 
research, In the initial development of an agency information system, 
however, there are a number of fairly simple approaches which can be 
taken. In this section of the guide, three of these methods will be 
briefly discussed - client satisfaction; goal attainment scaling and 


community adjustment follow-up questionnaires. 


CLIENT SATISFACTION 


Family Services of Hamilton-Wentworth employe fairly simple 
measurement of client satisfaction. Based upon their problem defini- 
tion system, caseworkers identify from one to three "target problems" 


upon which treatment will focus. 


At termination, the client and counsellor independently rate 
the extent of success attained in dealing with each of the problems. 


The rating is performed on a four point scale, as follows: 


Successful 
Partially successful 
Unsuccessful 


Unknown 


The Client Satisfaction Questionnaire utilized by the Harris 


County Mental Health and Mental Retardation Authority in Houston, Texas 


is presented in Table II. This goes slightly further than the previous 


approach insofar as it deals with specific aspects of the service 


provided. 
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TABLE II: HARRIS COUNTY CLIENT SATISFACTICN SCALE 


1. Did you have trouble: 


a) Making an appointment? 
b) Getting into a treatment program? 
c) Did you pay for services? 


d) Were agency personnel responsive? 


2. The treatment or services are helpful because of: 


a) Medication 
b) Specific advice received 
c) Increased self understanding 


ad) Adequate testing and proper placement 


3, Are you: 


a) Receiving good services? 
b) Satisfied with your therapist, counsellor or caseworker? 
c) Satisfied with the hours the unit is open? 


d) Do you prefer evening hours? 


4. If you had a choice, would you continue services at H.C. Mental 


Health and Mental Retardation Authority? 


NOTE: All items answered on a yes/no basis. 


Source: Justice, B., McBee, G "A Client Satisfaction Survey as One 


Element in Evaluation", Community Mental Health Journal, 
Vol. 14 (3), 1978, 248-252. 
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At a slightly more detailed level, the client satisfaction 
questionnaire from an American Health maintenance organization is 
presented in Table III, Although this questionnaire is designed for 
use in a health organization, most of the items, some with minor 


changes, could be applied to non-health oriented services. 


GOAL ATTAINMENT SCALING 


Goal attainment scaling (GAS) is a logical extension of 
problem oriented record keeping whereby client problems are translated 
into treatment goals and the level of goal attainment is subsequently 


monitored, 


In standard GAS format, each goal is scaled on five possible 
outcomes ranging from "best possible" to "worst possible" with 
corresponding numerical values ranging from +2 to ~2 with a mid-point 
of O (expected outcome). The clients level of attainment at intake is 
marked on the goal attainment form and, subsequently, at discharge and 
post-discharge follow-up, the outcome level is recorded. A goal 
attainment score is then calculated, based upon the changes between 


intake and discharge. 


More than one goal is typically utilized for each client and 
each goal can be given a numerical weighting which reflects its 
relative importance for the individual client. This weighting system 


can be included in the calculation of the goal attainment score. 


A standard goal attainment format, as utilized by a local 


youth services agency is presented in Table IV. 


For further reading, a goal attainment scale bibliography 
4s included in Appendix IV of this report. 
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TABLE III: CLIENT SATISFACTION SCALE 
All questions answered as "satisfied, neutral or dissatisfied". 


Technical knowledge, ability, competence of doctors. 
Personal interest and attention given by doctors. 


Attitudes and behavior of nurses, receptionists, cashiers, 
appointment clerks, other staff. 


Length of time between calling and date of appointment. 
Message center for contacting doctor. 


Amount of time spent on telephone with appointment or message 
center 


Hospital care 

Getting care in sn emergency 

Getting care without appointment 

Getting preventive care, e.g., physical exams. 
Amount of time spent with doctors 

Amount of explanation or information from doctors 
Costs in comparison to services covered 
Appearance, layout, design of clinics 


Location of facilities 


Source: Pope, C.R. "Consumer Satisfaction in a Health Maintenance 
Organization", Journal of Health and Social Behavior, Vol. 
19, 1978, 291-303. 
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COMMUNITY-BASED FOLLOW-UP QUESTIONNAIRES 


Traditional evaluation, particularly in the field of mental 
health has tended to concentrate upon changes in the psychological 
characteristics of the individual. Thus, for example, the Minnesota 
Multi-phasic Personality Inventory (MMPI) has often been utilized to 


measure changes in psychiatric symptoms as a result of treatment. 


Two major problems were associated with this approach. 
Firstly, tests such as the MMPI require highly trained personnel for 
interpretation. Unless a social service agency employed psychologists, 


this type of testing was not available to them. 


Secondly and, more importantly, the results of these psycho- 
logical tests did not indicate how the client actually functioned in 
the community after discharge. They only indicated how the client's 
personality or psychiatric symptons had changed. As a result of the 
emphasis of social service agencies upon "community adjustment", these 


tests were of limited utility. 


During the last decade, however, the rapid development of the 
community mental health movement and the subsequent growth of community 
mental health centres has resulted in the development of 4 number of 
fairly sophisticated community adjustment inventories. These tests are 
specifically designed to measure how the discharged client is function- 


ing in the community. 


One of the most popular of these scales, at present, is the 
Denver Community Mental Health Questionnaire. This seventy-nine item 
questionnaire measures eight major dimensions of community adaptation - 
psychological distress, isolation from family, isolation from friends, 
productivity, public system dependency, alcohol abuse, drug abuse and 


client satisfaction. 


ESATAMMOIT ZA) IW-WOdIO 


fatren ta bieft aut at psaleo eraq .aoissuleve rong it thet? 
iuptyolodsyay al! nt tegaaits wou stutineacse 4¢ habit aod itive . 
avowsnaiM aft ,siqtesn 101 ,eurt 4 «feubis tie at? 9s iterate stad | 
at hestitty! peek antto sad (IPM) yromevel ytii@werst otasdq- be ius 
trontieent ld tlunet 8 aa =item ofwdeidows 1 eouentle STiHem 
“2 

sitoneriq¢de wt} dtiw betaionesa tow amaidons ‘tolst owl 
tot Lenigerag haitkert yl hi atipet TIM oft ws rove mtaet , cheesey { 
seatgefodsysg Soyoiqns yousar ealyrer Iaioca » sale ,aeientomprad 
wad? of ofdafiars ton dow ynbteet Yo oayt eidd 


-oteyaq saedt to ativact ad? .“Eteerrogml stom .hng yihbrnsed 
i thiggno?? Yelfendas gontio ond tod otectin'® dor SP eteas Las tgot 
otiteelhin @4) wot hateoihe! vleo yer? .eprerosif tefte viicunoe ant 

wt? lo Jimeer 2 ah =, fogueds ke otras oittoidoyag so yt ienonseg 
rs , thw viinommoo” core yetorayue sorvTer Isscve i sfaadqna 
svitiivw betimit Yo stew arast 


rt} mn ae :  Getaet of? .<teravod ,sbross sek atf wobec 
- 7 
Bis ir , Sue mcus ad} fee Tiss 47S eaaA Leyes vt ieommoo . 


il » o’ delaeool > odd al Sotivee? aed eatins ciiead Ledase 


stay? gaadl ust tesver = ‘euthp ttre. betooitetitvon yittat 


a ‘» , anoles “3 Ys tealoqaq 4e22 af? Ve 4a0 

nibs ; »mifiinetfeasp ftirsH [utead vticunend sensed 

~ fottatqate x ion Ya in izumerh Toh sn tdyie eons, tien Leap 
einai wor? aalialont yeet mevt oniviisar yetettarh Iestyolosayng © 

' siniy wrth ,~rude iodoala ,qarebasqeh qeceys of iA cit ivitoehong : 


wmisostal ea deakis 
-_ 


- 18 


A second popular questionnaire for this purpose is the 
Community Adaptation Schedule, 2 217 item inventory which measures a 
number of community adjustment dimensions as well as individual 
affective, behavioral and cognitive traits. The community adjustment 
scales are work community, family community, social community, larger 


community, commercial community and professional community. 


This questionnaire, because of its considerable length, is 
somewhat more arduous to use than the Denver scale. Scoring is also 


considerably more complicated. 


References for these questionnaires are provided in Appendix 
IV of this report. 


SUCCESS AND NUMBERS 


To be used statistically, the evaluative information obviously 


has to be expressed numerically. 


Client satisfaction can be calculated as an "average 


satisfaction' score" based upon the responses to the questionnaire. 


Goal attainment scaling leads to a goal attainment score for 


each client. 


The community functioning questionnaires are usually used in 
a "before and efter" format. Each client has an average score on each 
scale at intake, another average score at discharge and, hopefully, 
another score three to six months after discharge as a follow-up. 
Thus, each client is administered the scale at least twice. The change 
in the client scores is the indicator of success. For example, one 
would hope for a decrease in the "Psychological Distress" score from 


the Denver Community Mental Health Questionnaire. 


Each one of these methods, then, can generate a clear 


numerical indicator of outcome or success. 
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V: LINKING THE COMPONENTS: THE DATA ANALYSIS 


The previous two chapters have dealt with the preparing of 
the information as the "inputs" of the information system. This 
information must now be analyzed to produce the proper summary docu- 


ments. 


Firstly, the information must be stored in a manner which 
ig amenable to analysis. If it simply remained in the client files, 
then it would be necessary to go through every file to obtain the 


necessary data. This is obviously impractical. 


For this reason, most agencies utilize a master sheet which 
contains all of the coded information. To ensure confidentiality, a 
"case number" is used instead of the client's name. This case number, 
which is a unique number assigned to the client at intake, also appears 
on the client file. This number is the only link between the master 


sheet and the file. 
An example of a simple master sheet is presented in Table V. 


Given a master sheet, the data can then be analyzed in a 
number of different ways. If the data is analyzed manually, then the 
calculations can be made directly from the sheet. If a computer is 
used, the data is usually transferred from the master sheet to a 
computer coding form which the key puncher uses to transcribe the data 


onto computer punch cards. 


A NOTE ON COMPUTERS 


If you do have access to a university computing centre, check 
into the cost of having your data computer analyzed. Very little 
programming sophistication is required. For the kind of analysis 
recommended in this guide, the "canned programs" included in the 
Statistical Package for the Social Sciences (SPSS) are more than 


adequate. Most university computing centres have these programs on hand. 
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The computer time itself is a minimal cost amounting to only 
a few dollars. You will need a programmer to "tell" the computer what 
SPSS program to use and what form you want the output to take. You 
will also need someone to do the keypunching. ‘Often you can find a 
graduate student or a faculty member who can do this for you either 


free or at a small cost, i.e. $100.00. 


THE MCBEE SYSTEM: A MANUAL COMPUTER 


The McBee Company has a system of punched cards which can be 
used to store the data and greatly eases the task of analysis, At 
least three agencies in Hamilton-Wentworth are using the McBee system 
with considerable success - Volunteer Bureau, Wesley House and Project 
LONAR. 


A sample McBee card is presented in Table VI. 


The data is transferred onto the cards by "opening up" the 
appropriate holes with a special punch. One thousand printed McBee 
cards cost between $200 and $250. This allows for the storage of data 
for 1,000 clients. 


ANALYZING THE DATA 


Whatever system you use, the analysis is relatively straight 
forward. Based upon the evaluative question discussed earlier in this 
guide, the purpose of the analysis is to discover which demographic 
and problem definition variables "predict" treatment success. The 
variables which are associated with success form the profile of the type 
of client which is most helped by your service. The evaluative data is, 


of course, your indicator of success. 


In the hypothetical example presented below, we will use 


"Nelient satisfaction" as our indicator of success. 
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TABLE VI 
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Firstly, you may want to ascertain whether the age of the 
client is associated with client satisfaction. Simply calculate the 
average client satisfaction score for all the clients in each age 
category. This will yield an "average success rate" for each age 
group. If the success rate is markedly higher in one group, then 


that age group appears to benefit more from treatment. 


This procedure can be repeated with each demographic variable 


as well as each category used in the problem-definition system. 


This is the most elementary form of analysis and more sophis- 
ticated forms can be used. For example, "cross-tabulation" can be 
utilized to identify if combinations of two variables are associated 
with success. Thus, for example, you may want to ascertain if "young 
marrieds" respond better than "young singles". At an even more complex 
level, a larger combination of demographic and problem oriented variables 
can be related to treatment success through "multiple regression tech- 
niques". As the level of sophistication increases, so does the need for 
computerization. If you do wish to use cross-tabulations or multiple- 


regression, consult with a qualified researcher. 


To begin, however, the simpler method described in the 
previous paragraphs is quite adequate. The results of this analysis, 
however, may suggest more questions which require a more sophisticated 


analysis. 
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APPENDIX I 


STATISTICS CANADA 
MAJOR OCCUPATIONAL CLASSIFICATIONS 


AcAMAD SOTYGITATS y 


SOTPAITTIS CALS JEMTeAT- 


mez 


I-1 
TWO DIGIT CODES 


Managerial, Administrative and Related 
Natural Sciences, Engineering and Mathematics 
Social Sciences and Related Fields 
Religion 

Teaching and Related 

Medicine and Health 

Artistic, Literary, Performing Arts 
Sports and Recreation 

Clerical 

Sales 

Service 

Farming and Argricultural 

Fishing, Hunting and Trapping 

Mining and Quarrying 

Processing 

Machining and Related 

Product Fabricating, Assembling and Repairing, Labour 
Construction Trades 

Transport Equipment Operating 

Meterial Handling 

Other crafts and equipment operating 


Occupations not elsewhere classified 
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A survey of seven large local agencies indicated 
that all of the agencies collected the following demographics: 


Some of 


1. 


Birthdate 

Sex 

Education 

Marital Status 
Occupation/Employment 


Income 


agencies also collected the following: 


Religion 

Family names and age 
Languege spoken 

Type of Residence 
Criminal Record 

Race 


Ethnic Origin 
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THREE LOCAL AGENCIES 


III-1 


FAMILY SERVICES OF HAMILTON-WENTWORTH 


DEFINITIONS: FORM 2 - PROBLEM IDENTIFICATION 


GENERIC PROBLEM DESCRIPTION 


The following describe the general types of problems as assessed by 
the worker. 


Interpersonal relationships : familial, includes relationship 
problems among any or all family members (adult partners, parent(s) 
and child(ren), etc.). 


Interpersonal relationships : extra familial, includes relationship 
problems between Srey of family and person(s) outside the 
family. 


Individual adjustment indicates that the problem (in terms of 
personality, behaviour or adjustment) lies within the individual 
client. 


Practical living skills implies that the problem area involves a 
lack of knowledge or inability to apply knowledge related to every 
day living exclusive of problems connected to financial management. 


Environment implies that the problem area is connected to or stems 
from societal, environmental or situational factors. 


Health and welfare implies that the problem area is specifically 
connected to, or stemming from, health or care factors. 


Financial management indicates that the problem area is connected 
to money management, such as use of credit, inability to meet © +7 
obligations to creditors, budgeting, priorities and planning. 


IDENTIFIED COMPONENTS 


The following describes the problem components as assessed by the 
workers after the intake phase. 


Emotional maladjustment/withdrawal. Includes such related problems 
as: difficulty in facing issues and in handling emotions involved 
with these issues; reactive or pervasive depression (excluding 
defined psychiatric illness); isolation (physical and/or emotional) ; 
and difficulty in articulating and communicating thoughts or 
emotions (including perceptural difficulties). 
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III-2 


Low_self-esteem. Beyond "Emotional maladjustment/withdrawal", 
indicates an acute sense of discouragement/failure, possibly 
combined with a low degree of motivation. 


Social contacts or use of leisure time, Destructive or 


inappropriate use of leisure time/or conflict in family as a 
result of this. 


Identified child abuse. Evidence recorded with a child welfare 
agency that parent has abused or neglected child, physically 
and/or emotionally. 


High risk for child abuse, The situation/emotional climate is 
such that child abuse or neglect is a high probability. 


Adjustment to separation/family breakup. Includes adjustment to 


marriage breakup, to child leaving home, parent leaving home or 
death in the family. 


Conflict concerning separation decision. Refers to problems in 
making decision to separate, 


Sexuality. Includes all problems (in terms of knowledge, 
behaviour or attitude) relating to sexuality. 


Pregnancy. Refers to any pregnancy which results from or 
produces conflict. 


Addiction. Includes conflict (external or internal, within group, 
society, or individual) caused by dependence on alcohol, drugs, 
or gambling (specified by worker. 


Occupational problems. Doing poorly at work, difficulty in 
holding a job, or being in an unsuitable job; excludes lack of 
available employment. 


School problems. Poor academic performance, truancy, misbehaviour, 
or poor peer relationships at school. 


(Potential) conflict with law. Includes delinquent/criminal/anti- 
social behaviour when the client has been in conflict with the law 
or there is a distinct possibility of this occurring. 


Parenting. Includes inconsistency in disciplining or rewarding 
children, conflict or rebellion; more generally this implies a 
lack of knowledge about the normal development (physical or 
emotional) of a child. 
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TII-3 


Domestic management, Includes problems with taking care of house, 
meals, inability to plan time realistically in this area, or lack 
of knowledge about nutrition. 


Money management. Client lacks the capacity to correlate income 
realistically to a comprehensive range of basic budgetary needs 
and obligations. As a consequence, income is allocated so 
inappropriately that certain basic needs and/or obligations to 
creditors are left inadequately provided for. 


Cultural adjustment, Refers to problems complicated by ethnic or 
regional background, including language-related difficulties 
within either individual or community. 


Accessibility of existing community resources. Includes lack of 
knowledge about existing community resources, or need for assist- 
ance in carrying out contact with resource. 


Inadequate education/training. Employment opportunities are 

Scarce or unsuitable because of lack of education, experience, and/ 
or job-related training or problems related to re-entry into work 
LOrce.. 


Restructured family. Family in which one or more of the parents 
have been previously involved jn another marriage/union; often 
involves children, extended family, etc. from both the present and 
previous unions, 


Unrealistic expectations. Client's aims/goals cannot be legiti- 
mately realized within the context of his/her circumstances 
(personal, environmental or financial). 


Poor Planning. Client lacks ability to foresee or to provide 
realistically for future (personal, environmental or financial). 


Change in circumstances. Significant change creates anxiety or 
conflict whether personal, environmental or financial. 


Poor occupational opportunities. Employment or training 
opportunities are scarce or unsuitable because of conditions 
existing within community. 


Inadequate services. Includes deficiency within community such as 


housing, health resources, social services, day care services, 
nursing homes or other services for the aged, recreational resources, 


transportation/mobility, or schools. 
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ITI-4 


Bureaucratic obstacles - community/government_ services. Refers 

to blockages encountered with particu:ar organization(s). Specify 
institution(s) concerned (including own organization, if applicable) 
and note potential scope of problem(s). 


Illness/disebility. Refers tc long-term or incapacitating illness, 
physical handicap or defined psychiatric illness necessitating 
psychiatric assessment and/or treatment. 


Inadesuate income, Lack of money for food, housing, medical 
expenses and other necessities. 
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ITI-5 
CHILDREN'S AID SOCIETY 


Reasons for Admission 


Emotional rejection 
Housing 
Imprisonment 

Lack of income 
Physical neglect 
Unfit home 

Lack of supervision 
Marital conflict 
No medical care 
Physical abuse 
Unwanted pregnancy 
Parent-child conflict 
Drug addiction 
Alcoholism 

Mental Illness 
Mental Retardation 
Physical Illness 
Sexual Offences 
Behaviour Problem 
Child Offender 
Court Order 
Deserted 

Orphan 


Truancy 
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III-6 
CHEDOKE CHILD AND FAMILY 


PRESENT PERCEIVED PROBLEM 


Classification of the patient's critical problem or problems at the 
time the patient is registered with CMHSIS, The various problem 
classifications are: 


School & Learning - problems concerning truancy, school 
phobia, specific learning disability, 
academic performance seriously below 
apparent ability. 


Thought & Perception problems related to difficulty in 
concentration, delusions, hallucina- 


7 ONSis 


Disruptive Behaviour - problems related to hyperactivity, 
boisterousness, chronic lying, stealing, 
running away, temper tantrums, property 
destruction, aggression towards others, 
non-compliance with adults, attention 
seeking through negative behaviour, 
sexual acting-out. 


Social Withdrawal - problems related to depression, withdrawal, 
lack of self-confidence, apathy, obsessive 
compulsive behaviour, secretive, self- 
inflicted injury or threat of same. 


Bodily Functions - problems related to sleep, eating problems, 
nervous mannerisms, eneuresis, encopresis, 
speech difficulties, drug and/or alcohol 
problems. 


Mental Retardation - any mental problems not specifically 
related to emotional mental problems. 


Other (Family) - any problems not mentioned in any of the 
other problem classifications. 
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